Video Processing In a Data Storage Device

ABSTRACT

A method and apparatus for video processing on a data storage device. A chip bound architecture includes a CMOS coupled to one or more NAND die, the CMOS including one or more processors, memories, and error correction code (ECC) engines capable of processing video data. According to certain embodiments, macroblocks are correlated between two I-frames, including motion vectors to define different locations of correlated macroblocks. A P-frame may be determined from a previous I-frame and its correlated macroblocks and motion vectors, while a B-frame may be determined from two or more adjacent I-frames with concomitant macroblocks and motion vectors, as well as P-frames associated with an adjacent I-frame.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of co-pending U.S. patent applicationSer. No. 17/349,515, filed Jun. 16, 2021, which is herein incorporatedby reference.

BACKGROUND OF THE DISCLOSURE Field of the Disclosure

Embodiments of the present disclosure generally relate to a data storagedevice, and more particularly, to a data storage device capable of videoprocessing.

Description of the Related Art

As generating a frame of video data from raw frame data is acomputationally expensive process, a subset of video frames arecalculated from pixel data, known as I-frames, representing periodframes throughout a video. Portions of I-frames, macroblocks, may becorrelated as between I-frames with different locations of correlatedmacroblocks in different I-frames defined by motion vectors. To delivera smooth video experience, intermediate frames known as B-frames andP-frames, between two I-frames may be generated using data from theI-frames, correlated macroblocks, and motion vectors.

Processing to generate a series of contiguous frames of video areconventionally carried out by a host retrieving I-frames, macroblocks,and motion vectors from a data storage device to generate intermediateframes. Computation of intermediate frames is typically accomplished ona host processor, such as a system CPU or one or more video cardprocessors.

Although the needed video data is stored on the data storage device,video processing such as described above conventionally is not carriedout on the data storage device, usage of host resources such as memoryand processors, in addition to requiring data transport of frame datafrom the data storage device to a processing destination.

What is needed are systems and methods for overcoming deficiencies ofprior approaches.

SUMMARY OF THE DISCLOSURE

The present disclosure generally to systems and methods for videoprocessing on a data storage device. A chip bound architecture includesa CMOS coupled to one or more NAND die, the CMOS including one or moreprocessors, memories, and error correction code (ECC) engines capable ofprocessing video data. According to certain embodiments, macroblocks arecorrelated between two I-frames, including motion vectors to definedifferent location of correlated macroblocks. A P-frame may bedetermined from a previous I-frame and its correlated macroblocks andmotion vectors, while a B-frame may be determined from two or moreadjacent I-frames with concomitant macroblocks and motion vectors, aswell as P-frames associated with an adjacent I-frame.

In one embodiment, a solid state drive is disclosed that includes amemory device, a first CMOS coupled to the memory device, and acontroller coupled to the memory device configured to executecomputer-readable instructions. When executed, the computer-readableinstructions cause the controller to receive at least a portion of oneof a first I-frame comprising a first macroblock and second I-framecomprising a second macroblock, on the memory device and correlate afirst macroblock to the second macroblock. The computer-readableinstructions further cause the controller to determine a first motionvector of the first macroblock and a second motion vector of the secondmacroblock, and encode a P-frame based on one of the first I-frame andthe first motion vector.

In another embodiment, a controller for a data storage device isdisclosed that includes a memory comprising computer-readableinstructions, and a processor configured to execute thecomputer-readable instructions that will cause the processor to store aportion of a first I-frame comprising a first macroblock on a jumboblock spanning multiple NAND dies, determine a motion vector of thefirst macroblock, and encode a P-frame based on the first I-frame andthe first macroblock.

In another embodiment, a data storage device is disclosed that includesa memory means configured to execute a method for video processing. Themethod includes receiving a first I-frame and a second I-frame at aplurality of CMOS CbAs, correlating the first I-frame and second I-frameto identify a first macroblock associated with the first I-frame andsecond macroblock associated with the second I-frame, the firstmacroblock correlated to the second macroblock, and calculating a firstmotion vector of the first macroblock and a second motion vector of thesecond macroblock.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above-recited features of the presentdisclosure can be understood in detail, a more particular description ofthe disclosure, briefly summarized above, may be had by reference toembodiments, some of which are illustrated in the appended drawings. Itis to be noted, however, that the appended drawings illustrate onlytypical embodiments of this disclosure and are therefore not to beconsidered limiting of its scope, for the disclosure may admit to otherequally effective embodiments.

FIG. 1 is a schematic block diagram illustrating a storage system inwhich a data storage device may function as a storage device for a hostdevice, according to certain embodiments.

FIG. 2A is a schematic illustration of dependencies of a group ofpictures of a video frame, according to certain embodiments.

FIG. 2B is a schematic illustration of an inter-frame predictionprocess, according to certain embodiments.

FIG. 3 is a schematic illustration of a Circuit Bounded Array (CbA)architecture, according to certain embodiments.

FIG. 4 is a schematic block diagram illustrating a controller managing aplurality of memory dies each including a CMOS chip of an NVM, accordingto certain embodiments.

FIG. 5 is a schematic block diagram illustrating a storage system inwhich a controller is configured to encode and decode a video frame,according to certain embodiments.

FIG. 6 is a schematic flow diagram illustrating a method of videoprocessing with multiple circuit chip architectures, according tocertain embodiments.

To facilitate understanding, identical reference numerals have beenused, where possible, to designate identical elements that are common tothe figures. It is contemplated that elements disclosed in oneembodiment may be beneficially utilized on other embodiments withoutspecific recitation.

DETAILED DESCRIPTION

In the following, reference is made to embodiments of the disclosure.However, it should be understood that the disclosure is not limited tospecifically described embodiments. Instead, any combination of thefollowing features and elements, whether related to differentembodiments or not, is contemplated to implement and practice thedisclosure. Furthermore, although embodiments of the disclosure mayachieve advantages over other possible solutions and/or over the priorart, whether or not a particular advantage is achieved by a givenembodiment is not limiting of the disclosure. Thus, the followingaspects, features, embodiments, and advantages are merely illustrativeand are not considered elements or limitations of the appended claimsexcept where explicitly recited in a claim(s). Likewise, reference to“the disclosure” shall not be construed as a generalization of anyinventive subject matter disclosed herein and shall not be considered tobe an element or limitation of the appended claims except whereexplicitly recited in a claim(s).

The present disclosure relates to systems and methods for videoprocessing on a data storage device. A chip bound architecture includesa CMOS coupled to one or more NAND die, the CMOS including one or moreprocessors, memories, and error correction code (ECC) engines capable ofprocessing video data. According to certain embodiments, macroblocks arecorrelated between two I-frames, including motion vectors to definedifferent location of correlated macroblocks. A P-frame may bedetermined from a previous I-frame and its correlated macroblocks andmotion vectors, while a B-frame may be determined from two or moreadjacent I-frames with concomitant macroblocks and motion vectors, aswell as P-frames associated with an adjacent I-frame.

FIG. 1 is a schematic block diagram illustrating a storage system 100 inwhich a host device 104 is in communication with a data storage device106, according to certain embodiments. For instance, the host device 104may utilize a non-volatile memory (NVM) 110 included in data storagedevice 106 to store and retrieve data. The host device 104 comprises ahost DRAM 138. In some examples, the storage system 100 may include aplurality of storage devices, such as the data storage device 106, whichmay operate as a storage array. For instance, the storage system 100 mayinclude a plurality of data storage devices 106 configured as aredundant array of inexpensive/independent disks (RAID) thatcollectively function as a mass storage device for the host device 104.

The host device 104 may store and/or retrieve data to and/or from one ormore storage devices, such as the data storage device 106. Asillustrated in FIG. 1 , the host device 104 may communicate with thedata storage device 106 via an interface 114. The host device 104 maycomprise any of a wide range of devices, including computer servers,network attached storage (NAS) units, desktop computers, notebook (i.e.,laptop) computers, tablet computers, set-top boxes, telephone handsetssuch as so-called “smart” phones, so-called “smart” pads, televisions,cameras, display devices, digital media players, video gaming consoles,video streaming device, or other devices capable of sending or receivingdata from a data storage device.

The data storage device 106 includes a controller 108, NVM 110, a powersupply 111, volatile memory 112, the interface 114, and a write buffer116. In some examples, the data storage device 106 may includeadditional components not shown in FIG. 1 for the sake of clarity. Forexample, the data storage device 106 may include a printed circuit board(PCB) to which components of the data storage device 106 aremechanically attached and which includes electrically conductive tracesthat electrically interconnect components of the data storage device106, or the like. In some examples, the physical dimensions andconnector configurations of the data storage device 106 may conform toone or more standard form factors. Some example standard form factorsinclude, but are not limited to, 3.5″ data storage device (e.g., an HDDor SSD), 2.5″ data storage device, 1.8″ data storage device, peripheralcomponent interconnect (PCI), PCI-extended (PCI-X), PCI Express (PCIe)(e.g., PCIe x1, x4, x8, x16, PCIe Mini Card, MiniPCI, etc.). In someexamples, the data storage device 106 may be directly coupled (e.g.,directly soldered or plugged into a connector) to a motherboard of thehost device 104.

Interface 114 may include one or both of a data bus for exchanging datawith the host device 104 and a control bus for exchanging commands withthe host device 104. Interface 114 may operate in accordance with anysuitable protocol. For example, the interface 114 may operate inaccordance with one or more of the following protocols: advancedtechnology attachment (ATA) (e.g., serial-ATA (SATA) and parallel-ATA(PATA)), Fibre Channel Protocol (FCP), small computer system interface(SCSI), serially attached SCSI (SAS), PCI, and PCIe, non-volatile memoryexpress (NVMe), OpenCAPI, GenZ, Cache Coherent Interface Accelerator(CCIX), Open Channel SSD (OCSSD), or the like. Interface 114 (e.g., thedata bus, the control bus, or both) is electrically connected to thecontroller 108, providing an electrical connection between the hostdevice 104 and the controller 108, allowing data to be exchanged betweenthe host device 104 and the controller 108. In some examples, theelectrical connection of interface 114 may also permit the data storagedevice 106 to receive power from the host device 104. For example, asillustrated in FIG. 1 , the power supply 111 may receive power from thehost device 104 via interface 114.

The NVM 110 may include a plurality of memory devices or memory units.NVM 110 may be configured to store and/or retrieve data. For instance, amemory unit of NVM 110 may receive data and a message from controller108 that instructs the memory unit to store the data. Similarly, thememory unit may receive a message from controller 108 that instructs thememory unit to retrieve data. In some examples, each of the memory unitsmay be referred to as a die. In some examples, the NVM 110 may include aplurality of dies (i.e., a plurality of memory units). In some examples,each memory unit may be configured to store relatively large amounts ofdata (e.g., 128 MB, 256 MB, 512 MB, 1 GB, 2 GB, 4 GB, 8 GB, 16 GB, 32GB, 64 GB, 128 GB, 256 GB, 512 GB, 1 TB, etc.).

In some examples, each memory unit may include any type of non-volatilememory devices, such as flash memory devices, phase-change memory (PCM)devices, resistive random-access memory (ReRAM) devices,magneto-resistive random-access memory (MRAM) devices, ferroelectricrandom-access memory (F-RAM), holographic memory devices, and any othertype of non-volatile memory devices.

The NVM 110 may comprise a plurality of flash memory devices or memoryunits. NVM Flash memory devices may include NAND or NOR based flashmemory devices and may store data based on a charge contained in afloating gate of a transistor for each flash memory cell. In NVM flashmemory devices, the flash memory device may be divided into a pluralityof dies, where each die of the plurality of dies includes a plurality ofphysical or logical blocks, which may be further divided into aplurality of pages. Each block of the plurality of blocks within aparticular memory device may include a plurality of NVM cells. Rows ofNVM cells may be electrically connected using a word line to define apage of a plurality of pages. Respective cells in each of the pluralityof pages may be electrically connected to respective bit lines.Furthermore, NVM flash memory devices may be 2D or 3D devices and may besingle level cell (SLC), multi-level cell (MLC), triple level cell(TLC), or quad level cell (QLC). The controller 108 may write data toand read data from NVM flash memory devices at the page level and erasedata from NVM flash memory devices at the block level.

The power supply 111 may provide power to one or more components of thedata storage device 106. When operating in a standard mode, the powersupply 111 may provide power to one or more components using powerprovided by an external device, such as the host device 104. Forinstance, the power supply 111 may provide power to the one or morecomponents using power received from the host device 104 via interface114. In some examples, the power supply 111 may include one or morepower storage components configured to provide power to the one or morecomponents when operating in a shutdown mode, such as where power ceasesto be received from the external device. In this way, the power supply111 may function as an onboard backup power source. Some examples of theone or more power storage components include, but are not limited to,capacitors, super-capacitors, batteries, and the like. In some examples,the amount of power that may be stored by the one or more power storagecomponents may be a function of the cost and/or the size (e.g.,area/volume) of the one or more power storage components. In otherwords, as the amount of power stored by the one or more power storagecomponents increases, the cost and/or the size of the one or more powerstorage components also increases.

The volatile memory 112 may be used by controller 108 to storeinformation. Volatile memory 112 may include one or more volatile memorydevices. In some examples, controller 108 may use volatile memory 112 asa cache. For instance, controller 108 may store cached information involatile memory 112 until the cached information is written to the NVM110. As illustrated in FIG. 1 , volatile memory 112 may consume powerreceived from the power supply 111. Examples of volatile memory 112include, but are not limited to, random-access memory (RAM), dynamicrandom access memory (DRAM), static RAM (SRAM), and synchronous dynamicRAM (SDRAM (e.g., DDR1, DDR2, DDR3, DDR3L, LPDDR3, DDR4, LPDDR4, and thelike)).

Controller 108 may manage one or more operations of the data storagedevice 106. For instance, controller 108 may manage the reading of datafrom and/or the writing of data to the NVM 110. In some embodiments,when the data storage device 106 receives a write command from the hostdevice 104, the controller 108 may initiate a data storage command tostore data to the NVM 110 and monitor the progress of the data storagecommand. Controller 108 may determine at least one operationalcharacteristic of the storage system 100 and store the at least oneoperational characteristic in the NVM 110. In some embodiments, when thedata storage device 106 receives a write command from the host device104, the controller 108 temporarily stores the data associated with thewrite command in the internal memory or write buffer 116 before sendingthe data to the NVM 110.

The data storage device further includes an encoder/decoder 118. Theencoder/decoder 118 is coupled to the buffer 116 and the NVM 110, suchthat data is encoded prior to being programmed to the NVM 110 anddecoded after the data is read from the NVM 110, but before beingreturned to the controller 108. It is to be understood that theencoder/decoder 118 may be separate components of the data storagedevice 106.

FIG. 2A is a schematic illustration of dependencies of a plurality ofpictures 202-226 of a video frame 200, according to certain embodiments.The video frame 200 may be received by a host device, such as the hostdevice 104 of FIG. 1 . Each picture of the plurality of pictures 202-226is either an I-frame, a B-frame, or a P-frame. I-frames include anentire image and are coded without reference to any other frame. In oneembodiment, I-frames may be programmed in SLC since I-frames may beaccessed more frequently than B-frames and P-frames.

P-frames use previous frames to be decoded and may include image data,motion vector displacements, and combinations thereof. P-frames predictthe current frame from an earlier picture, such as an I-frame or aB-frame. B-frames use previous and future frames for decoding of theB-frame, such that B-frames are bi-directionally predicted frames. Forexample, in the video frame 200, a first picture 202 (I-frame) is usedto predict or encode a fourth picture 208 (P-frame). The first picture202 and the fourth picture 208 are used to predict or encode a secondpicture 204 (B-frame) and a third picture 206 (B-frame). In someembodiments, each picture of the plurality of pictures 202-226 is storedon separate, but adjacent memory dies. For example, the first picture202 is stored on die0, the second picture 204 is stored on die1, thethird picture 206 is stored on die2, and so on.

FIG. 2B is a schematic illustration of an inter-frame prediction process250, according to certain embodiments. The video frames are divided intomacroblocks. When encoding the macroblocks, the encoder may determine ifthere is a block similar to the currently encoded block on a previouslyencoded frame. Because adjacent frames in a video frame, such as thevideo frame 200, are typically similar, motion vectors are used to pointto the position of the matching block from the reference frame. Forexample, in FIG. 2A, the first picture 202 is used to predict the fourthpicture 208, where the first picture 202 and the fourth picture 208 aresimilar. In reference frame 252, a motion vector points to the bestmatch with regards to the target macroblock of the target frame 254.Although the reference macroblock found may not be an exact match to thetarget macroblock, an encoder computes differences between them using atleast one of Discrete Cosinus Transform, Quantization, RLE, and Huffmancode. Therefore, when decoding the encoded frame, the decoder is able todetermine the raw pixels of the macroblock.

A macroblock of the target frame 254 is correlated to a macroblock ofthe reference frame 252. Although the macroblock of the target frame 254may not be identical to the reference frame 252, the calculated motionvectors are used to shift the reference frame 252 to match the targetframe 254. The encoding of a P-frame and a B-frame may utilize thecalculated motion vectors and the target frame 254, where the targetframe 254 is an I-frame.

FIG. 3 is a schematic illustration of a Circuit Bounded Array (CbA)architecture 300, according to one embodiment. The CbA architecture 300includes one or more complementary metal oxide semiconductor (CMOS)chips 306 coupled to respective one or more memory arrays 308 via one ormore connection units. In one embodiment, each memory array 308comprises a memory die. The CMOS chips 306 may include an encoder unitand a decoder unit as well as similar processing logic as a controller,such as the controller 108. The CMOS chips 306 may further include anerror correction code (ECC) engine, which may be part of the encoderunit and the decoder unit. The CMOS chips 306 may be configured toschedule received write commands to program data to the coupled memorydie, and schedule received read commands to read data from the coupledmemory die. It is to be understood that that NAND dies may be referredto as memory dies for exemplary purposes.

In the description herein, it is to be understood that CMOS chips andCbA may be used interchangeably for exemplary purposes, and embodimentsdescribed herein may also be applicable to CMOS Under Array (CuA)architectures and CMOS Above Array (CAA) architectures.

FIG. 4 is a schematic block diagram 400 illustrating a controller 402managing a plurality of memory dies 406 a-n each, including a CMOS chipof an NVM stack 404, according to certain embodiments. The NVM stack 404may be the NVM 110 of FIG. 1 . The controller 402 may be a controller ofa host device, such as the host device 104 of FIG. 1 , or a controllerof the data storage device, such as the controller 108 of FIG. 1 . Thecontroller 402 manages the video processing multi-CbA, such as the CbAarchitecture 300, in the memory compute system. Each CMOS chip, such asthe CMOS chip 306 of FIG. 3 , of each memory die of the plurality ofmemory dies 406 a-n are configured to handle the computational tasks,such as determining the motion vectors between frames. The controller402 may handle the video processing, such as the encoding and decodingof the B-frames and the P-frames.

I-frames, B-frames, and P-frames may be processed in parallel in-memory,in each CbA (e.g., the plurality of CMOS chips 306). Furthermore,because I-frames, B-frames, and P-frames may be atomically processed ona single memory die, a minimal amount of transfers of results betweendies may occur. Furthermore, the autocorrelation calculation of aB-frame and a P-frame of each die may be completed in parallel by eachrespective CbA using motion vectors of adjacent frames. Motion vectorshave a size less than the size of a frame, such that CbA side channelsmay handle the transfer of motion vectors between CbAs.

The CbA to CbA side channels may allow for reducing duplicatedcalculations performed by the data storage device. For example, theautocorrelation calculation between two frames on two dies may be splitbetween two CbAs. By splitting the autocorrelation calculation betweentwo CbAs, the computation effort may be reduced by about 50%.Furthermore, the calculated motion vectors of macroblocks of adjacentmemory dies or relatively close memory dies may be used to calculate themacroblocks of the next memory dies. The macroblocks of video framesthat have a high count value (i.e., used more often) may be distributedvia the side channels of the CbAs so that adjacent CbAs may use the morecommonly used macroblocks in the computation of the autocorrelationcalculations.

FIG. 5 is a schematic block diagram illustrating a storage system 500 inwhich a controller 504 is configured to encode and decode a video frameaccording to certain embodiments. A host device 502, such as the hostdevice 104 of FIG. 1 , is coupled indirectly to the controller 504. Thehost device 502 sends data, such as a video, to controller 504. Thecontroller 504 includes a tunable ECC module 506 and a motion vectorsunit 514. The controller 504 is coupled to the NVM 522, where the NVM522 includes a plurality of memory dies with CMOS chips 524 a-n. In someexamples, the controller 504 divides the tasks of frame encoding betweenthe plurality of memory dies 406 a-n and the associated CbAs.Furthermore, controller 402 may tune the error correction code (ECC)code rate per frame, per die, and/or per CbA according to the frame type(e.g., I-frame, P-frame, and B-frame).

The tunable ECC module 506 includes a format check 508, a tunable ECCencoder 510, and a tunable ECC decoder 512. The format check 508 maydetermine the format of the video frame, such that the picture frame isassociated with a frame type. Based on the format of the video frame,the tunable ECC encoder 510 may determine an amount of ECC data toencode the video frame with. For example, if the format of the videoframe is an I-frame, then the video frame may be programmed with agreater amount of ECC data than that of a B-frame or a P-frame. Forexample, the ECC parity protection of I-frames may be increased whilereducing the code rate since the data of an I-frame is more significantwith higher replication rates when compared to other dependent frames.Code rates may be increased to allow for decreased ECC data protectionand decreased for increased ECC data protection. The amount of ECC dataprotection may be based on the usage of the relevant data or the numberof copies stored of the relevant data. B-frames and P-frames may haveregular or even reduced ECC protection since the replication rate ofB-frames and P-frames are low. Alternatively, in a system that storesduplicate I-frames across multiple dies, the other copies of the sameI-frame may be used to recover failures when reading an I-frame on aspecific die. Thus, the ECC data may be reduced on I-frames due to thestorage of duplicate I-frames.

In one embodiment, the tunable ECC encoder 510 may be tuned according tothe calculated motion vector sizes of the macroblocks and the predictederror information, such that macroblocks with small motion vector valueshave a smaller number of ECC data and macroblocks with high motionvector vales have a larger number of parity bits. In another embodiment,the number of usages for macroblocks with small motion values may becollected, and more ECC data may be generated for the macroblocks thathave a high usage number. Because macroblocks with small motion valuesmay be used more often, more ECC data may be required since there may bea large dependency on the macro blocks with the small motion values.Furthermore, the tunable ECC decoder 512 may be configured to decode thereceived encoded data from the plurality of memory dies 524 a-n.

The motion vectors unit 514 includes a format check 516, a frame encoder518, and a frame decoder 520. Because video encoding of B-Frames andP-frames may require a large amount of bandwidth, such that the encodingis a large computational task, duplicate I-frames may be stored onseparate memory dies. Therefore, the B-Frames and the P-frames ofmultiple dies may be encoded and decoded in parallel using the relevantCMOS chips and the relevant duplicate I-frame.

The format check 516 may be similar to the format check 508. The frameencoder 518 encodes the video frame to be stored to the NVM 522 and theframe decoder 520 decodes the video frame retrieved from the NVM 522.The frame encoder 518 may encode a calculated or determined motionvector in the video frame, where the motion vector is calculated basedon an adjacent or nearby macroblock.

In one embodiment, each die of the plurality of dies 524 a-n receives afull frame to process (using the relevant CMOS chip). The processedframe and the related I-frame are located on the memory die. In anotherembodiment, two frames may be split between multiple dies of theplurality of dies 524 a-n as part of a jumboblock. A jumboblock is alogical grouping of blocks. The controller 504 is configured to organizethe data of the jumboblocks, such that an autocorrelation calculationdata is available in each die, and data transfers between dies and CbAsare minimized. For example, upon detecting that an I-frame resides inLBA ranges that span multiple dies as part of a jumboblock, thecontroller 504 may duplicate information in each die or reorder datasuch that the physical location of the I-frame is optimally placed in adie so that multiple dies may access the I-frame concurrently. In yetanother embodiment, the controller 504 program macroblocks fully in asingle die in order to enable easier correlation calculation.

FIG. 6 is a schematic flow diagram illustrating a method 600 of videoprocessing with multiple circuit chip architectures, according tocertain embodiments. For exemplary purposes, the method 600 may beexecuted by the controller 504 of FIG. 5 or the controller 402 of FIG. 4. At block 602, a data storage device, such as the data storage device106 of FIG. 1 , is provided. The data storage device includes aplurality of NVM dies have a CbA architecture, such as the plurality ofmemory dies 406 a-n each including a CMOS chip of FIG. 4 . At block 604,the controller calculated an autocorrelation of two video frames usingthe CbA chip coupled to an NVM die. For example, the autocorrelation mayinclude determining a similarity between the two video frames.

At block 606, the controller identifies common macroblocks andcalculates their motion vectors. The motion vectors may be calculated bya CbA. For example, a first CbA may determine a first motion vector fora first macroblock and a second CbA may determine a second motion vectorfor a second macroblock. The identification and the calculation may bethe inter-frame prediction process 250 of FIG. 2B. For example, theidentification may include determining that a reference macroblockmatches a target macroblock. After determining that a referencemacroblock matches a target macroblock, the calculation is completed todetermine a shift position between the reference macroblock and thetarget macroblock based on the motion vectors of each of themacroblocks.

At block 608, the calculated motion vectors are shared between the CbAsusing the side channels of the CbAs in order to aid and speed up othercalculations using a common reference macroblock or video frame, such asan I-frame. At block 610, a B-frame or a P-frame is encoded using theI-frame and the calculated motions of the macroblocks.

By processing video frames using CMOS chips coupled to memory dies,efficient parallel video encoding and decoding processes may be achievedby sharing computed data between the CMOS chips and decreasingduplicated computations. Furthermore, the calculation state parametersare completed by the CMOS chips coupled to the memory dies, which mayallow for operation suspending and resuming without losing data orhaving to back up data.

In one embodiment, a solid state drive is disclosed that includes amemory device, a first CMOS coupled to the memory device, and acontroller coupled to the memory device configured to executecomputer-readable instructions. When executed, the computer-readableinstructions cause the controller to receive at least a portion of oneof a first I-frame comprising a first macroblock and second I-framecomprising a second macroblock, on the memory device and correlate afirst macroblock to the second macroblock. The computer-readableinstructions further cause the controller to determine a first motionvector of the first macroblock and a second motion vector of the secondmacroblock, and encode a P-frame based on one of the first I-frame andthe first motion vector.

The instruction that causes the controller to correlate the firstmacroblock to the second macroblock is executed by the first CMOS. Thefirst macroblock and second macroblock include an image. The solid statedrive further includes computer-readable instructions to cause thecontroller to calculate a shift of position of the image as between thefirst macroblock and second macroblock based on one of the first motionvector and second motion vector. The solid state drive further includesa second CMOS. The computer-readable instructions that cause thecontroller to determine the first and second motion vectors furthercause the first motion vector to be determined on the first CMOS, andthe second motion vector to be determined on the second CMOS. Thecomputer-readable instructions further cause the controller to encode aB-frame based on at least one of the first I-frame, second I-frame, andP-frame. The first CMOS includes an ECC engine. The computer-readableinstructions further cause the controller to adjust one of a code rateor parity bits, based on one of the first I-frame, second I-frame,P-frame, and B-frame.

In another embodiment, a controller for a data storage device isdisclosed that includes a memory comprising computer-readableinstructions, and a processor configured to execute thecomputer-readable instructions that will cause the processor to store aportion of a first I-frame comprising a first macroblock on a jumboblock spanning multiple NAND dies, determine a motion vector of thefirst macroblock, and encode a P-frame based on the first I-frame andthe first macroblock.

The computer-readable instructions further cause the processor to storea second I-frame comprising a second macroblock, and autocorrelate thefirst macroblock with the second macroblock. The computer-readableinstructions further cause the processor to store the first macroblockand second macroblock on a NAND die. The computer-readable instructionsfurther cause the processor to reorder the first macroblock and secondmacroblock to physically locate each on the NAND die, or duplicateinformation of the first macroblock and second macroblock and cause theduplicate information to be physically located on the NAND die. Thecomputer-readable instructions further cause the processor to optimizeECC performance of a tunable ECC based on at least one of the firstI-frame, second I-frame, and P-frame. The computer-readable instructionsfurther cause the processor to optimize ECC performance of a tunable ECCbased on a size of the motion vector. The computer-readable instructionsfurther cause the processor to maintain a count of usage of one of thefirst I-frame, second I-frame, or P-frame, and optimize ECC performanceof a tunable ECC based on the count.

In another embodiment, a data storage device is disclosed that includesa memory means configured to execute a method for video processing. Themethod includes receiving a first I-frame and a second I-frame at aplurality of CMOS CbAs, correlating the first I-frame and second I-frameto identify a first macroblock associated with the first I-frame andsecond macroblock associated with the second I-frame, the firstmacroblock correlated to the second macroblock, and calculating a firstmotion vector of the first macroblock and a second motion vector of thesecond macroblock.

The method further includes distributing one of the first motion vectorand second motion vector between at least two CMOS CbAs of the pluralityof CMOS CbAs. The data storage device further includes encoding one of aB-frame or a P-frame based on one of the first I-frame and the secondI-frame. Each of the plurality of CMOS CbAs includes a tunable ECC. Themethod further includes tuning the tunable ECC of at least one of theplurality of CMOS CbAs based on one of the first I-frame and secondI-frame. The method further includes further tuning the tunable ECC ofat least one of the plurality of CMOS CbAs based on one of the firstmotion vector and second motion vector.

While the foregoing is directed to embodiments of the presentdisclosure, other and further embodiments of the disclosure may bedevised without departing from the basic scope thereof, and the scopethereof is determined by the claims that follow.

What is claimed is:
 1. A solid state drive, comprising: a memory device; a first complementary metal oxide semiconductor (CMOS) coupled to the memory device; and a controller coupled to the memory device configured to execute computer-readable instructions that when executed cause the controller to: receive at least a portion of one of a first I-frame comprising a first macroblock and second I-frame comprising a second macroblock, on the memory device; correlate a first macroblock to the second macroblock; determine a first motion vector of the first macroblock and a second motion vector of the second macroblock; and encode a P-frame based on one of the first I-frame and the first motion vector.
 2. The solid state drive of claim 1, wherein the instruction that causes the controller to correlate the first macroblock to the second macroblock is executed by the first CMOS.
 3. The solid state drive of claim 2, wherein the first macroblock and second macroblock comprise an image, further comprising computer-readable instructions to cause the controller to calculate a shift of position of the image as between the first macroblock and second macroblock based on one of the first motion vector and second motion vector.
 4. The solid state drive of claim 2, further comprising a second CMOS, wherein the computer-readable instructions that cause the controller to determine the first and second motion vectors, further cause the first motion vector to be determined on the first CMOS, and the second motion vector to be determined on the second CMOS.
 5. The solid state drive of claim 4, wherein the computer-readable instructions further cause the controller to encode a B-frame based on at least one of the first I-frame, second I-frame, and P-frame.
 6. The solid state drive of claim 1, wherein the first CMOS comprises an ECC engine.
 7. The solid state drive of claim 6, wherein the computer-readable instructions further cause the controller to adjust one of a code rate or parity bits, based on one of the first I-frame, second I-frame, P-frame, and B-frame.
 8. A data storage device, comprising: a memory means configured to execute a method for video processing, the method comprising: receiving a first I-frame and a second I-frame at a plurality of complementary metal oxide semiconductor (CMOS) circuit bounded arrays (CbAs); correlating the first I-frame and second I-frame to identify a first macroblock associated with the first I-frame and second macroblock associated with the second I-frame, the first macroblock correlated to the second macroblock; and calculating a first motion vector of the first macroblock and a second motion vector of the second macroblock.
 9. The data storage device of claim 8, the method further comprising distributing one of the first motion vector and second motion vector between at least two CMOS CbAs of the plurality of CMOS CbAs.
 10. The data storage device of claim 9, further comprising encoding one of a B-frame or a P-frame based on one of the first I-frame and the second I-frame.
 11. The data storage device of claim 10, wherein each of the plurality of CMOS CbAs comprises a tunable ECC.
 12. The data storage device of claim 11, the method further comprising tuning the tunable ECC of at least one of the plurality of CMOS CbAs based on one of the first I-frame and second I-frame.
 13. The data storage device of claim 12, the method further comprising further tuning the tunable ECC of at least one of the plurality of CMOS CbAs based on one of the first motion vector and second motion vector.
 14. A data storage device, comprising: a memory device; and a complimentary metal oxide semiconductor (CMOS) coupled to the memory device, wherein the CMOS is configured to: receive a first frame and a second frame; determine a reference macroblock between the first frame and the second frame; calculate a first motion vector for the first frame and a second motion vector for the second frame, wherein the calculating is based on the reference macroblock; and encode either a B-frame or a P-frame using the calculated first motion vector, the calculated second motion vector, and the reference block.
 15. The data storage device of claim 14, wherein the first frame and the second frame are I-frames.
 16. The data storage device of claim 14, wherein the CMOS is further configured to share the calculated first motion vector, the calculated second motion vector, and the reference macroblock with an adjacent CMOS.
 17. The data storage device of claim 16, wherein the CMOS and the adjacent CMOS share a side channel, and wherein the sharing occurs via the side channel.
 18. The data storage device of claim 16, wherein the adjacent CMOS is configured to utilize the shared calculated first motion vector, the shared calculated second motion vector, and the reference macroblock to encode P-frames and B-frames of the adjacent CMOS.
 19. The data storage device of claim 14, wherein the encoding is different between the B-frame and the P-frame.
 20. The data storage device of claim 14, wherein calculating the first motion vector for the first frame and the second motion vector for the second frame further comprises determining a shift position between the reference macroblock and each of the first frame and the second frame. 